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Optimal Dynamic Contracts for a Large-Scale 
Principal-Agent Hierarchy: A Concavity-Preserving Approach 

Christopher W. Miller* Insoon Yangl 


Abstract 

We present a continuous-time contract whereby a top-level player can incentivize a hierar¬ 
chy of players below him to act in his best interest despite only observing the output of his 
direct subordinate. This paper extends Sannikov’s approach from a situation of asymmetric 
information between a principal and an agent to one of hierarchical information between several 
players. We develop an iterative algorithm for constructing an incentive compatible contract 
and define the correct notion of concavity which must be preserved during iteration. We identify 
conditions under which a dynamic programming construction of an optimal dynamic contract 
can be reduced to only a one-dimensional state space and one-dimensional control set, inde¬ 
pendent of the size of the hierarchy. In this sense, our results contribute to the applicability of 
dynamic programming on dynamic contracts for a large-scale principal-agent hierarchy. 


1 Introduction 

A principal-agent problem is a problem of optimal contracting between two parties in an uncertain 
environment. One player, namely the agent, may be able to influence the value of the output 
process with his actions. A separate party, the principal, wants to maximize the output process, 
which is subject to external noise. 

It would be ideal for the principal to monitor the agent and enforce a strategy which is optimal 
to the principal. This case if often called the first best. However, it is often costly for the principal 
to monitor the agent’s action. In many practical situations, the principal can only observe the 
output process, while the agent has perfect observations. Therefore, the agent is free to deviate 
from the action or control suggested by the principal, and the principal may not be able to detect 
this deviation. For example, the agent can shirk and attribute the resulting low output to noise. 

In this setting of asymmetric information, the principal can only enforce an ^incentive compatible^ 
control strategy upon the agent - a strategy which maximizes the agent’s expected utility given 
the compensation scheme in the contract. Then, the principal’s optimal strategy is to design 
the combination of a compensation scheme and a recommended control strategy such that (i) 
the recommended control is optimal to the agent given the compensation scheme, and (ii) this 
combination maximizes the principal’s expected utility. Such a combination constitutes a contract, 
which is provided to the agent. 
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This so-called second best strategy in principal-agent problems has been one of primary inter¬ 
ests in economic studies on contracts [12) . In particular, principal-agent problems in continuous¬ 
time have attracted considerable attention in economics, finance and engineering (e.g.,[IOlll 2 l| 26 l). 
Mathematically, such a principal-agent problem can be considered as a special case of Stackelberg 
differential games [53]. While many Stackelberg differential game problems remain as a challenge 
in general |5] , there have been remarkable advances in solution approaches for principal-agent prob¬ 
lems. 

In Sannikov [201121] . a dynamic programming approach is developed. The key idea is to introduce 
another state variable, called the agent’s continuation value, which represents the agent’s expected 
future utility conditioned upon the information up to current time. This approach is generalized 
in |5] to handle more complicated dynamics of the output process and multiple agents. Another 
stream of research utilizes the stochastic maximum principle iiniiiis|. This method requires us 
to solve an associated forward-backward stochastic differential equation, which is a non-trivial task 
(e.g., miTHIEI). 

In this paper, we consider a hierarchical principal-agent problem with asymmetric information 
in the form of moral hazard. This means there exists a hierarchy of players, each of which acts as 
a ‘principal’ to the ‘agent’ below them in the hierarchy. 


Po Pi Pfc —>■ Pfc+i Pat- 

More precisely, in this hierarchy of players. Player k acts as a principal with Player fc -|- I as his 
agent. Moral hazard refers to the restriction that Player k can only compensate Player fc -I- I based 
upon an observable noisy output, but not upon the actions actually performed by Player fc -I- I. Due 
to usefulness of contractual hierarchies in modeling a organizational form of firms, governments and 
vertically-integrated economies, several contracts in a principal-agent hierarchy have been proposed 
(e.g., [HITiKTi]). However, these studies focus on incentive contracts in static settings. A recent 
work [22] extends Holmstrom and Milgrom’s optimal linear contract [10] to the case of discrete-time 
hierarchical contracting by using first-order conditions. To the authors’ best knowledge, our work 
first develops a dynamic programming solution approach for designing optimal continuous-time 
dynamic contracts in a principal-agent hierarchy. 

The main result in continuous-time moral hazard problems between two players is that the 
principal can provide a payment scheme which incentivizes the agent to act in the principal’s best 
interest. In [20ll6]. the success of these payment schemes depends upon certain concavity conditions 
on the agent’s utility. The particular conditions and how to proceed when they are not met were 
emphasized in [5]. 

The main idea of this paper is to construct a payment scheme for the hierarchical principal-agent 
model, whereby Player 0 can incentivize all players below him to act in his best interest. At a high 
level, the idea is to think of all players below Player 0 as an ‘aggregate agent’ and inductively apply 
a generalized two player principal-agent result. 

Principal —>■ 

Po ^ 


The first contribution of this work is to illustrate how Player 0 can construct an optimal dynamic 
contract which incentivizes the entire hierarchy to act in his best interest. Compared to m, this 
result allows Player 0 to indirectly incentivize even those players whose output he cannot directly 
observe. Specifically, Player k can only monitor the output of Player fc -|- I for fc = 0,1, • • • , — I. 


Aggregate Agent 

Pi Pfe+i P/v 
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The main technical issue in this result is that, in general, the effective utilities of each player in 
the hierarchy will not be concave. To restore concavity, we develop a new concept called (/, F)- 
concave envelopes, which play an essential role to design an optimal dynamic contract via dynamic 
programming. 

Secondly, this work illustrates how our hierarchical principal-agent model can resolve dimension¬ 
ality issues in dynamic contract problems with multiple agents. Compared the contract constructed 
by a single principal controlling multiple agents directly in [S], Player 0 only monitors one contin¬ 
uation value in a dynamic programming solution of his optimization problem. Furthermore, the 
maximization step is performed over a subset of efforts with dimension potentially less than N. 
We show how to construct this subset by a novel iterative algorithm. This iterative construction 
allows us to characterize a stochastic optimal control problem, which is equivalent to the original 
contract design problem. An important feature of this stochastic optimal control problem is that 
it is one-dimensional and therefore an optimal dynamic contract can be designed by solving an 
one-dimensional dynamic programming problem. 

The final contribution of this work is to characterize conditions under which the subset of efforts 
which Player 0 can choose is a one-dimensional sub-manifold. In particular, we show that if each 
effort has a utility function which is strictly concave, then we can parameterize the admissible 
efforts by a function which we construct. In this case, we show Player 0 can construct an optimal 
dynamic contract by solving an associated Hamilton-Jacobi-Bellman (HJB) equation with only one¬ 
dimensional state space and one-dimensional admissible controls. We explicitly demonstrate this 
HJB characterization in the case of agents with quadratic utilities. 

The rest of this paper is organized as follows. In Sectionj^ we provide the mathematical setting 
of the hierarchical principal-agent problem considered in this paper. In Section we define a 
generalized two player principal-agent problem and provide an optimal dynamic contract for the 
principal. This contract is iteratively constructed in Section to provide an optimal dynamic 
contract for the hierarchical principal-agent problem. In Section we consider more details about 
the main concavity property of each agent’s utility which much be preserved during iteration, 
called (/, r)-concavity. In particular, we prove a major result related to dimensionality reduction. 
Finally, in Section]^ we consider a specific example and illustrate the steps involved in constructing 
an optimal dynamic contract for the hierarchical model. 

2 Problem Formulation 

Consider a hierarchy of -I- 1 players, each with a principal-agent relationship. We summarize the 
information asymmetry in the following diagram: 


Fq —>■ Pi Pfe Pfc+i —>■•••—>■ Pat. 

Note that Player k can only monitor the output of Player fc -|- 1 for fc = 0,1,-- - ,A^—1. Because 
the output is affected by external noise. Player k is not able to correctly infer Player fc -I- I’s action. 
For example, if output is low, Player k cannot deduce if it is because Player fc -|- 1 performed little 
effort or due to bad luck. Each player provides payment to the player below him in the hierarchy, 
but the payment can only depend upon observable outputs and not the actual effort performed. 

The goal of this paper is to define a payment mechanism whereby Player 0 can incentivize all 
players below him to act in a prescribed manner, even though Player 0 cannot directly observe 
their actions. The proposed payment mechanism is in the form of a contract between the principal, 
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Player 0, and the aggregate agent, Players 1, • • • ,N. The contract consists of a compensation 
scheme and a recommended control strategy for each agent, i.e.. Player A: for fc = 1, • • • ,N. These 
two components must be designed before the contract starts. Player O’s interest is to design a 
contract such that (A) given the compensation scheme each agent follows the recommended control 
strategy, and (ii) this combination maximizes Player O’s expected utility. 


2.1 Mathematical Setup 

We consider a continuous-time system representing the output of each player up to a terminal time, 
r > 0. Let (f2, P, P) be a probability space supporting a standard Brownian motion B with natural 
filtration We also consider output processes Xk with dynamics given below. We denote by 

(t) the filtration generated by the process Xk ■ 

The dynamics of the system are as follows: 




Player N chooses an effort process Apf which is adapted to . The output of Player is a 
process defined by 

dXjq = Ajqdt -I- adB. 


• For fc = 1, • • • , — 1, Player k chooses an effort process Ak adapted to and terminal 

compensation Rk+i which is (T)-measurable. The output of Player fc is a process driven 

by 


dXk — Akdt dXk+i 

= {Ak Aj^) dt odB. 


( 1 ) 


Intuitively, Xk represents the cumulative output from effort by Player k and those below in 
the hierarchy. 

• Player 0 chooses a terminal compensation scheme i?i which is (T)-measurable. 


The utilities of the players are as follows: 


• Player N has some cost for effort, and receives a terminal compensation from Pn-i- 


[An,Rn] :=E^ 



{A]\[') dt Rn 


• For k = I,-- - ,N — Player k has some cost for effort, receives a terminal compensation 
from Player k — 1, and pays a terminal compensation to Player fc -|-1: 

[ (Ak) dtRk{Rk+i) ■ 

0 

Note that the Player fc’s utility can depend on Ak+i-.N as well as Ak because Ak is adapted 
to and Xk+i is driven by Q which depends on Ak+i-.N- In other words, the utility 

for Player k depends upon effort processes for Player A: -|- 1 through Player N because these 
processes affect the distribution of Xk+i^ which affects the distribution of Ak directly. 


[Ak-.N, Rk, ^/c+i] 
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• Player 0 has utility from the output Xn and pays a terminal compensation to Player 1: 


rT 

J°[A^.,n,R,]:=E^ / dX^ + q°iR,) 

Jo 

f Ai + • • • + Ajqdt + cP (i?i) 
Jo 


= 


The notation Ak-.N represents the tuple (A^, • • • , Aat). When fc = 1, we will often use the notation 
A to denote the tuple (Ai, • • • , A^v)- The notation E'^ clarifies that we are taking expectation 
under a measure where the dynamics of Xj. are evolved via the choice of effort A. The set of 
admissible control for Player k is given by 

Ak '■= {Afc : [0,r] —>■ M I Afc(t) is (t)-progressively measurable} 

for A: = 1, • • • ,N —1 and An '■= {An : [0,T] —> M | AnP) is -progressively measurable}. The 
set of admissible compensation for Player k is given by 

TZk '■= {Rfc € K I Rfe is (T)-measurable} 

for A: = 2, • • • , A and TZi := {Ri e M | Ri is (T)-progressively measurable}. We also let 
A := Ai X ■■■ X An- Analogous notation is used for TZ. 

In this paper we assume that, for each 1 < A; < A, the function is concave and is decreasing 
and linear. Economically, these assumptions correspond respectively to aversion to uncertainty in 
effort and risk-neutrality over terminal payments. 


2.2 Economic Problem 

The goal of Player 0 is to prescribe compensation schemes, ,Rn)j which allow him to 

choose the effort for every player, {Ai, ■ ■ • ,An)- In particular, he does this so as to maximize his 
expected utility. However, because Player 0 cannot directly observe efforts, the choice must satisfy 
an incentive compatibility condition - that Player k would not be better off (i) by providing Player 
A;-|-l a different compensation scheme Rk+i or (ii) by performing a different effort A{. Furthermore, 
Player 0 cannot force other players to accept a contract, so the choice must satisfy an individual 
rationality condition - that Player k is paid more than a minimum required to accept the contract. 
We let Wfc € M be the minimum required utility for Player k. 
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Player O’s optimization problem can then be formulated as follows: 

sup J°[Ai:N,Ri] 

AcA,RC'R 

subject to J^[Ai:n,Ri,R 2 ]> J^[A[,j^,Ri,R 2 ] R 2 ) £ Ai-.n x R 2 

J^[Ai:N, Rl, R 2 ] > Wi 

j'^iAk-.NT Rk, Rk + l] > j’^iA'k-N, Rk, R'k + l] '^{A'k:Ny ^'k+l) G Ak-.N X R-k+l 
j'^[Ak-.N, Rk,Rk+l] > Wfc 

J^[An,Rn] ^ [Aj^, Rn] '^Aj^ £ An 

J^[An, -Rw] > Wat, 

where the restrictions represents incentive compatibility and individual rationality conditions for 
the other players. 

In summary, Player 0 proposes a set of recommended effort processes A £ A and terminal 
compensations R £ TZ in order to maximize his expected utility. The first two constraints refer to 
Player I’s incentive compatibility and individual rationality respectively - for fixed i?i, the proposed 
choices must maximize Player I’s expected utility and meet a minimum utility for him to accept 
the contract. Each pair of constraints after that represent the analogous incentive compatibility 
and individual rationality constraints for each Player k. 

As written, Player O’s optimization problem is non-trivial to solve primarily because it is unclear 
how to deal with the various incentive compatibility constraints. We will show that these constraints 
can be replaced by restrictions on the form of the terminal payments R and admissible suggested 
efforts. Furthermore, the reformulation will be proven to be exact. 

2.3 Main Result 

The main result of this paper is to extend the continuous-time formalism described for two player 
Stackelberg differential games by Sannikov [501 this hierarchical setup. We will show that for 
a particular choice of payment schemes, (Ri, • • • ,Rn), the incentive compatibility conditions will 
automatically be satisfied. 

We will show that Player O’s optimization problem (§ is equivalent to the following optimization 
problem: 

sup J°[A, i?i] 

AeA,Yiey 

s.t. A{t) € P C t £ [0,r] a.e. 

Yi(t) £ (d^) (Ai(t) -f • • • -f Ajv(t)), t G [0, T] a.e. 

for a particular choice of (Ri,-- - ,Rn) € R, a given concave function ^ : K —>• K, and a given 
subset P C , which will be characterized. Here, d(j) represents the super-gradient of (j), and 
y := {Y : [0,r] —>■ M I Y(t) is R-®(t)-progressively measurable}. 

The intuition is that Player 0 has an optimal effort process A in mind. Player 0 provides Player 
1 with a terminal compensation i?i, which incentivizes him to cause the rest of the hierarchy to 
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follow A = [Ax, ■ ■ ■ ,An). The function 0 represents the effective cost that Player 1 has to cause 
the hierarchy to follow A, and Yi represents the marginal cost. 

There is not an explicit form for T and 0, but the iterative algorithm defined in Theorem is 
constructive. In Section 5, we show that if all players have utilities which are strictly concave, then 
T is a one-dimensional manifold. 

The main idea of the construction of the contract is to iteratively apply a generalization of a 
two player principal-agent result. The main technical condition that this generalization features 
over those in [eiEo] is /-concavity, which was highlighted in [8]. When dealing with generalized 
dynamics for the output process, as in the hierarchical case, /-concavity is the key condition an 
agent’s utility must satisfy for a principal to incentivize him to follow a given effort. We will show 
how this concavity condition can be preserved as we iteratively apply the two player principal-agent 
result. This process leads to constraints on {Ai, ■ ■ ■ ,An), which are encoded in the subset T. 

This simplified problem is a standard stochastic control problem which may be solved by dynamic 
programming or, equivalently, by solving an associated Hamilton-Jacobi-Bellman equation. Then, 
Player 0 can explicitly construct a dynamic contract which is incentive compatible for all players, 
satisfies all individual rationality constraints, and maximizes his expected utility. 

In particular, define the value function of Player 0 as 


v{w, t) 


sup 

AcA*,YiCy* 


(>T ^ 


Ak{s)ds (Ri) \W{t) =w 


k=l 


where yl* := n {A : [0, T] ^ K | A{t) G T, t e [0, T] a.e.}, j;* := /; n {Y : [0, T] ^ K | Y{t) G 
(00) (l^A(t)) ,t G [0,T] a.e.}, and W{t) solves the following stochastic differential equation: 

dW (t) = —p,i {A{t))dt — aY {t)dB. 


The drift function /ri : > K is defined in the iteration algorithm as a linear combination of each 

player’s utility function. The constant cr is the volatility of Xi. The interpretation of W{t) is the 
continuation value for Player 1 - his remaining expected utility from following the proposed effort 
conditioned on the information up to time t. 

Then, v is the unique viscosity solution of the following Hamilton-Jacobi-Bellman (HJB) equa¬ 
tion: 

Vt sup 

aG r 1 

VC{d4>i)(Y a) 

with terminal condition v{w,T) = q{w), where Pi C T will be characterized in the following 
sections. After solving the HJB, Player 0 can then construct an optimal contract by constructing 
an optimal feedback control A*{t) G Ti and Y*{t) G (00)(l^A*(t)) starting from the initial value 
W{Q) which will be specified. Furthermore, the optimal feedback control will be adapted to the 
correct filtrations. Therefore, the proposed contract is implementable. 


y (7 Vww Pi{.Oj)v-u) J- 1 a 


= 0 , 


3 Contract for Generalized Principal-Agent Problem 

In this section, we consider a generalized principal-agent problem between two players. We propose 
a contract and demonstrate its incentive compatibility and optimality. We will ultimately iterate 
this result in the full principal-agent hierarchy in Section]^ 
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Compared to uniiH], this problem features generalized dynamics, lack of concavity on the agent’s 
utility, and constraints on what efforts the agent may choose. All three of these features will arise 
naturally in our iterative construction of a contract for the principal-agent hierarchy. 


3.1 The Setting 

As in the hierarchical model, we consider a continuous-time system with an underlying probability 
space (n, P) supporting a standard Brownian motion B with filtration . There are also two 
output processes and A^r with dynamics defined below. We also take special consideration of the 
filtrations and which represent those generated by the processes Xa and X^^ respectively. 

In this problem, the single agent chooses an n-dimensional effort process A = (Ai,--- ,A„), 
which is restricted to be A'®-adapted and lie in a prescribed subset T C M" almost surely. Mean¬ 
while, the principal chooses a one-dimensional effort process P, which must be -adapted, as 
well as a terminal payment to the agent, Ra, which must be (r)-measurable. The principal 
also receives a terminal payment, i?,r, which is (T)-measurable - the interpretation is that this 
is a utility from a player higher in the hierarchy. 

The dynamics of X^ and A^r are given by 

dXa = f{A)dt adB, 
dA^n. = Pdt dXa ■ 


The utilities for each party are given by 

Wo + / r°'{A)dt P Ra , 

Jo 


The principal’s optimization problem can then be written as follows: 

sup J’’’ [P, A,Ra,RTr] 

PCV ,AcA,RaC:Ti 

subject to J°‘[A, Ra] > J°‘[A', Ra] VA'G A (^ 3 ^ 

A(t) G T, tG[0,r]a.e. 

J“[A, i?a] > Wq,, 

where A := {A : [0,T] —>• K | A(t) is A^(<)-progressively measurable}, "P := {P : [0,T] — 
K I P(t) is (t)-progressively measurable} and IZ := {R G K | R is (T)-measurable}. 

Intuitively, this problem represents a principal-agent problem with generalized dynamics and 
utilities. The agent chooses a multi-dimensional effort process to determine his output process. 
On the other hand, the principal chooses a one-dimensional effort process and a terminal payment 
based upon the agent’s output process and some external noise. 

The requirement that each process be adapted to different filtrations represents the information 
asymmetry in the problem in the form of moral hazard. The relationship with our hierarchical 
problem will be that the principal represents Player k, while the agent represents Player k + 1 
controlling the efforts of all those players below him in the hierarchy. 


(^P, X^dt P (^RfA P R^ 


J“[A,i?A :=E^ 
JAP,A,R^,RA :=E^ 
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3.2 /-Concavity and (/, r)-Concave Envelopes 

In this subsection, we define the key concavity condition required for the proposed contract to be 
incentive compatible and optimal. 

Definition 1. Let f : > K". We say a function g : K™ —)■ K is f-concave if there exists a 

concave function (j) : K" —)■ K such that 

9{x) = Vx e K™. 

This condition was shown in [5] to be key to incentive compatibility in standard principal- 
agent problems. In particular, it was shown that if the agent’s utility was not /-concave, then the 
principal is restricted to particular values of effort depending on the so-called f-concave envelope 
of the agent’s utility. 

In this paper, as we iterate along the hierarchy, there will always be restrictions on the set of 
admissible efforts. Therefore, we need a generalized notion of concave envelopes. 

Definition 2. Let g : K"* —?> M 6e some function and T C R"*. We say g : R™ —>■ R is the 
(/, r)-concave envelope of g if 

• g is f-concave, 

• 9{x) > 9 {x) for all a; S T, and 

• If g is an f-concave function such that g(x) > g{x) for all a: G T, then g{x) > g(x) for all 

X G r. 

When convenient, we may refer to g by (jj o f. 

We show in Section 5 how to construct the (/, r)-concave envelope of an arbitrary function. 
The equivalence g = 4> o f refers to an auxiliary concave function / produced by the construction. 
We also explicitly define the touching set to be the subset where g = g. 

Definition 3. Let g be the {f,T)-concave envelope of g. We define the corresponding touching set 
to be: 

r' := {x G r : gix) = g{x)} . 

An example of the touching set is shown in Figure [l] Finally, we recall the definition of the 
super-gradient HIT] of a concave function. 

Definition 4. Let cj) : R” —>■ R &e a concave function. For any x G R", define the super-gradient 
of (j) at X as the following set: 

d4>{x) := {y G R” : /(x) -I- {x — x) > 4>{x) Vx' G R"} . 
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Figure 1: g is the (/, r)-concave envelope of g when f{x) = x and F = M. F' is the corresponding 
touching set. 


3.3 The Contract 

We claim that for any desired choice of {Ai, • • • , A„), there exists a compensation which incen- 
tivizes the agent to follow the desired effort. Furthermore, it is optimal for the principal to choose 
a contract of this particular form. 

Let f°' = (j>ofhe the (/, r)-concave envelope of r“, and let 

F' := {(oi,- • • ,a„) e F I f“(ai, • • • ,a„) = r“(ai,- • • ,a„)} 

be the corresponding touching set. We also introduce an auxiliary control variable Y which 
plays an important role in designing a compensation that satisfies the incentive compati¬ 
bility and individual rationality. The set of admissible T’s is given by y := {Y : [0,T] —> 
M I Y(t) is J'®(f)-progressively measurable}. 

Proposition 1. Fix P &P. For any choice of processes A G A such that A{t) G F' for t € [0,T] 
a.e., and Y such that Y(f) € (df)) {f{A{t))) fort € [0,r] a.e., define 

f{A)Y-r^{A)dt- rYdX^. (4) 

Jo 

Then, A is incentive compatible with Ra, i.e. 

JAARa\ > yA' e .4n{A : [0,T] A(t) G r,t G [0,r] a.e.}, 

and also satisfies the agent’s individual rationality constraint, i.e. 

.J°’[A,Ra] > Wa. 

Proof. Suppose that the agent instead follows an arbitrary choice of effort A' = {A},-- - ,A'A) G 


Ra ■= W„ - Wo 
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An {A : [0,T] —>■ M I A{t) GT,t & [0,T] a.e.}. Then, we compute 


:= E 


A' 


Wo + / r°‘{A')dt Ro 
Jo 


-E 


A' 


< Wq, + E^ 

< w„. 


[ r“(A')-f“(A) + /(A)rdt- [ YdX, 
Jo Jo 

r - f^{A) + Y [f{A) - f{A')] dt 
Jo 


The key in this argument is that if the agent follows A' instead of A, then the process Xa evolves 
as 

dXa = f{A')dt + adB^ , 

where is a -Brownian motion. The existence of the new probability measure P^ is guaran¬ 
teed by the Girsanov theorem (e.g., [H]). The rest follows from concavity using A(t) G T', f“ > r“, 
^ o /, and the definition of d(j). 

If the agent follows the suggested effort A = (Ai, • • • , A„) e A n {A : [0, T] —^ M | A{t) GT',t G 
[0, T] a.e.}, we have 


J“[A,i?A:=E^ wo-f [ r°‘{A)dt + R^ 

Jo 

[ r“(A)-f“(A)-h/(A)rdt- [ YdX^ 
Jo Jo 

f r^[A)-f°‘{A)dt 
Jo 


= w„ -I- E 


= w„ -I- E 


= w„ > J“[A',i?A VA' e An {A : [0,T] ^ K | A{t) GT,tG [0,T] a.e.}. 


Here, the last equality holds due to the fact that A{t) is in the touching set T' for t G [0,T] a.e. 
Therefore, A is incentive compatible with R^ and the individual rationality condition is satished. □ 


Note that R^ given by 0 is not an admissible compensation scheme because it may not be 
(y)-measurable. In fact, it is J^^(T)-measurable. However, we will show in the following 
subsection that any contract optimal to the principal has Ra such that it is (T)-measurable 
and therefore is implementable. 


3.4 Optimality for Principal 

In the previous subsection, we proved that a class of contracts is incentive compatible and individ¬ 
ually rational for the agent. In this subsection, we prove that it is sufficient for the principal to 
consider only contracts of this form. In other words, the proposed class of contracts is optimal to 
the principal. 

First, we prove two inequalities which relate r“ to its (/, r)-concave envelope = (j) o f and 
the super-gradient d(j). 
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Lemma 1. Let = 4>o f be the {f,T)-eoncave envelope of r°‘ and recall 

r' := {aer|r“(a) = f“(a)}. 

Then, for any a G F, we have 

Vy ^ (/(q))> then there exists o' € F such that 

f“(a) + y(/(a')-/(o))<r“(o'). 

2. If y & {dcf) if (a)) and a ^ F', then there exists o' G F such that: 

r“(a) + y(/(a')-/(o))<r“(o'). 


Proof. Both proofs proceed by contradiction by building a better (/, F)-concave envelope than 
f°‘=(j)of, 

1. First, let y ^ {d(j)) (/(a)). Then, there exists o' € F such that 

f“(a) + y(/(a')-/(o))<f“(o'). 

Furthermore, suppose that in fact for all o' G F we have 

f“(o) + y(/(a')-/(o))>r“(a'). 

Define a function : M" —>■ K as 


f“(a;) := min{f“(a;), f°'{a) + yif{x) - /(o))} . 


Then, we have 

> r“. 

Therefore, f“ is /-concave as the minimum of two /-concave functions. Flowever, at o', we 
have 

f“(a') = f“(a) + yifia') - /(a)) < f“(a'). 

Flowever, this contradicts the fact that is the (/, F)-concave envelope. Therefore, there 
exists o' G F such that 

f“(o) + y(/(a')-/(o))<r“(a'). 

2. Next, let y G {dcf) {f [a)), but a ^ F'. Then, r“(o) < f“(a). Furthermore, suppose that in 
fact for all o' G F we have 


r“(o) + y(/(a')-/(o))>r“(a'). 
Define a function : M" —)■ K as 


f“(a;) := min{f“(a:), r“(o) +y{f{x) - /(o))} 


Then, we have 


> r“. 

Therefore, f“ is /-concave as the minimum of two /-concave functions. However, at o, we 
compute 

f“(a) = r“(a) < f“(o). 

This is contradictory to the fact that f“ is the (/, F)-concave envelope. Therefore, there 
should exist o' G F such that r“(a) -I- j/(/(a') — /(a)) < r“(a'). 
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□ 

Next, we demonstrate that any incentive compatible contract must be in the form of that defined 
above. This will suggest that it is in fact optimal for the principal to choose a contract of this form. 

Proposition 2. Suppose that A G AD {A : [0,T] —>■ M | A{t) G T,t G [0,T] a.e.} is incentive 
compatible with Ra G TZ. Then, the following must hold: 

R^=w-G f f{A)Y-r°‘{A)dt- [ YdX^ (3) 

Jo Jo 

for some constant ic G K, where Y Gy such that Ylt) G (d^) {f{A{t))) fort G [0,T] a.e. 

Proof. Let A G An {A : [0,T] —>• K | A(t) G T,t G [0,T] a.e.} and Ra G TZ satisfy the incentive 
compatibility constraint. Define 

Vit) := 

By the Martingale representation theorem (e.g., [mils] ), there exists a unique (up to a set of zero 
measure) process Y Gy such that 

V{t)-\- f r°‘{A)ds = w— f OaYdB 
Jo Jo 

= w f f{A)Yds— f YdXa, 

Jo Jo 


r°‘{A)ds + Ra I X^{t) 


where w = y(0). Then, we have 


Ra = V{T) 

= w+ [ f{A)Y -r°‘{A)dt- 
Jo 

Next, we show that Y{f) G {f{A{t))) for t G [0,T] a.e. Suppose that Y{t) ^ {dcf) {f{A{t))) 
for t G 71, where 7} is a set of a strictly positive P x A-measure. By the first claim in Lemma we 
can find A' G Ac\ {A : [0,T] —)■ K | A(t) G L,t G [0, T] a.e.} such that 

f“(A(f)) + Y{t) [/(A'(t)) - f{Am < r“(A(f)) Vt G Ti. 
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Then, the agent would prefer to follow A! over A because 


=E 


A' 


wo+ [ r°‘{A')dt Ra 

Jo 


= Wo + w + E‘ 


A' 


> Wo + w + E- 


> Wo + w + E- 


A' 


A' 


r {A)+Y[f (A) 

Jo 

[ r“(A')-f“(A)+y[/(A) 
Jo 

[ r°‘{A')-r°‘{A')dt 
Jo 


f{A')] dt 
fiA')]dt 


= Wo + w 

= r[A,R^]. 


But this contradicts the incentive compatibility of the contract, so Y{t) £ (dj)) {f{A{t))) for t G 
[ 0 ,T] a.e. □ 

The compensation given solely by (|^ is -measurable and may not be in In Theorem 

we will design an optimal contract in a relaxed space such that ^ holds, and show that at an 
optimum the compensation is in TZ and therefore is implementable. 

Next, we show that the principal should only consider contracts with the suggested effort such 
that A{t) £ r' for t £ [0,T] a.e. Any other suggested effort would not be incentive compatible. 

Proposition 3. For any incentive compatible contract, we have 

A £ A n {A : [0, T] ^ K I A{t) £ T', t £ [0, T] a.e.}, 

where 

r' := {(oi, • • • , a„) £ r I • • • , a„) = r“(ai, • • • , a„)} . 

Proof. By Proposition the compensation of any incentive compatible contract must be of the 
form ^ 

f{A)Y - r°‘{A)dt - [ YdXa, 

Jo 

with Y Gy such that Y (t) £ (8$) {f{A{t))) for t £ [0, T] a.e. given A £ An {A : [0, T] —)■ M | A(t) £ 
r,t £ [0,r] a.e.}. Suppose that A{t) ^ T' for t £ 72, where 72 is a set of a strictly positive P x A- 
measure. By Lemma we can find A" £ A n {A : [0,r] —)■ K | A(t) £ r,t £ [0,T] a.e.} such 
that 

r“(A(t)) + Y{t) [f{A"{t)) - /(A(t))] < r“(A"(0) Vt £ %. 
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But we can check, the agent would prefer to follow A" over A since 




Wo + / r°‘{A”)dt Ra 


= Wo + w + E 

> Wq + W 


A" 


r - r“(^) + y if (A) - f{A")] dt 

Jo 


But this contradicts the incentive compatibility of the contract, so {Ai{t),--- ,An{t)) G F' for 
t G [0, T] a.e. □ 

Finally, because is strictly decreasing, we can argue that it suffices to consider contracts with 
w = Wa — Wq. This choice makes the individual rationality constraint binding. Any larger choice 
of w would be sub-optimal for the principal. 

Up to this point, we have made no reference to the payment that the principal receives at the 
terminal time, which can be quite general. We have restricted the class of admissible compensation 
schemes for the agent, but which of the remaining the principal chooses will depend upon In 
particular, the principal will solve an optimization problem summarized in the following theorem: 

Theorem 1. Consider the following optimization problem: 


sup J^[P,A,Ra,RT,] 

Pcv,AeA,Ycy 

subject to A{t) S r' C M", t G [0,T] a.e. 

Y{t)G{d^){f{Am, tG[0,T]a.e., 


where Ra is given by 


Ra='Wa- Wo 


f f{A)Y-T^{A)dt- [ YdX^. 
Jo Jo 


( 5 ) 


Then, the optimization problem @ is equivalent to the original contract design problem @. 
Proof. By Propositions and incentive compatible contracts Ra are characterized by 

f{A)Y - r^{A)dt - [ YdXa 
Jo 



for u; G M, A S ^ such that A{t) G F' C M" and Y G y such that Y(t) G {d(j>) (/(A(t))) for t G [0, T] 
a.e. By the same computation as in Proposition!^ we observe that Ra is individually rational for 
the agent if and only if 

J“ [A, Ra] = Wq -G W > Wa- 

Now suppose Ra has w > w^ — wq. Then, we define a new utility, 

f{A)Y - r°‘{A)dt - [ YdXa. 

Jo 


R'a=^a- Wo 
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Note that both {A,Ra) and {A,R'^) satisfy the incentive compatibility and individual rationality 
conditions. However, we have 

r [P, A, R'^, R^] = r [P, A, R^,R^J + lq^(R'J - q^(Ra)J 
> r [p,H,p„,p^], 

because q^ is strictly decreasing and R'^ — Ra = Wq, — Wq — w < Q almost surely. Therefore, we 
conclude it is optimal for the principal to choose w = — Wq. 

Let A* and Y* solve the optimization problem Q . We now show that Ra given by (§ with A* 
and Y* is in 1Z and therefore is implementable. The problem Q can be rewritten as the following 
stochastic optimal control problem: 

sup [P, A, Pq , P^] 

Pev,AeA,Yey 

subject to dW = —r°‘{A)dt — YdB 

= {f{A)Y -r^{A))dt-YdXa 

W{0) = Wq, — Wq 
Ra = W{T) 

fe[o,r]a.e. 

r(<)G W)(/(H(f))), tG[0,T]a.e., 

which can be solved by dynamic programming. Its solution is given by a feedback control, (H*, Y*) = 
{A*{W{t)),Y*{W{t))). Since W is adapted to so are A* and Y* . Therefore, Pq given by ([^ 
with A* and Y* is (T)-measurable and therefore is in TZ. □ 

In summary, we have converted a quite general two-player principal-agent with individual ratio¬ 
nality and incentive compatibility constraints into an equivalent stochastic optimal problem with 
only point-wise restrictions on the values of the effort process. The idea in the remainder of the 
paper will be to iterate this choice with explicit choices of P^r to solve Player O’s problem in the 
principal-agent hierarchy. 


4 Contract for Principal-Agent Hierarchy 

Our goal here is to inductively apply the result from above. The hope is that Player 0 can design 
compensation Pi, • • • , Piv in such a way that Players 1 through Player N will find it optimal to 
follow the desired efforts. 

Principal -A 

Po ^ 


Theorem 2. Suppose that q^{r) = —fikf for k = 1, - ■ ■ ,N — 1. Let 

N N 

P-k{Ak-.N) ■■= ^ yk,jC{Aj), := ^ 

j=k j=k 


Aggregate Agent 

Pi Pfe Pfe+i —>•••—> P/v 


( 6 ) 
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with yk,k '■= 1, 7fcj := Yle^t A; J = ^ ’ ’ ’ ) for k = 1, ■ ■ ■ , N — 1. Let be the concave 

envelope ofr^ = and := {a^ G K|r-^(a7v) = 4>n{o,n-)}. Set pk as (Afc + - • • + Ajv, Rx F^+i)- 
concave envelope of pk o-nd F/j C as the corresponding touching set for k = N — 1, - ■ ■ ,1. 

Consider the following optimization problem: 

sup J°[A, i?i] 

AcA,Yicy 

subject to A{t) G Fi C , t G [0,T] a.e. C) 

Yi{t) G {dpi) {Ai{t) + • • • + Afi[{t)), i € [0,T] a.e., 

where ^ 

{Ai + • • • + Ajsf) Yi — Pi (yl) dt — f YidXi. 

Jo 

Then, the optimization problem Q is equivalent to the original contract design problem ([^. 

The intuition behind this result is that by choosing a compensation scheme of the form Ri, 
Player 0 can incentivize every player below him in the hierarchy to follow a ‘suggested effort’ 
process A = {Ai, ■ ■ ■ ,Aiy). However, it is only possible for Player 0 to incentivize the hierarchy to 
follow efforts A{t) S Fi. 

The function pi : —>■ K represents the cost to Player 0 for incentivizing the hierarchy to 

perform A. The constant w* represents a minimum expected utility Player 0 must provide to 
satisfy the entire hierarchy’s individual rationality contraints. Intuitively, the function pi encodes 
information about incentive compatibility conditions. 



Proof. We proceed by induction. In the base case, we fix a choice of Rn-i- Player TV — I will then 
solve the following optimization problem: 


sup ^[An-i, An, Rn-i, Rn] 

-1\N 

subject to J^[An, Rn] > J^[A'n, Rn] ^A'j^gAn 
J^[An^ Rn] > wjv. 


This falls in the form of the generalized two player principal-agent problem. Therefore, we can 
define the compensation Rn as 

Rn:=-wn+ [ AnYn — r^{AN)dt — f YnJXn, 

Jo Jo 

where Yn G y such that YN{t) G dpN{AN{t)) for t G [0,T] a.e., and pN is the standard concave- 
envelope of r^. Define the set 

Tn := {un G R I r^{aN) = pN{aN)}- 
Due to Theorem [l] it is equivalent for Player — 1 to solve 

sup J^~^[An-i, An, Rn- 1 , Rn] 

AN-l:NCiAjV-l..N 

YNey 

subject to AN{t) G Tn, tG[0,T] a.e. 

YN{t) G dpN{AN{t)), t G [0,r] a.e. 
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Next, we consider the inductive step. For fixed Rk, Player k solves the following optimization 
problem: 


sup J'^[Ak-.N,Rk,Rk+i] 

^k\N^Ak:N 
Rk + l:N^'^k + l:N 

subject to J^+^[Ak+i:N,Rk+l,Rk-i- 2 ] > ^k+i, R'k+ 2 ] 

'^^'k+l-.N € Ak+l-.N, R'k+2 € 'Rk+2 
j'^~^^[Ak+l-.N, Rk+1, Rk+l] > Wfc+i 

J^[An, Rn] > J^lA'j^, Rn] e An 
J^[An, Rn] > Wat. 


The inductive assumption is that it is equivalent for Player k to solve 


sup j’^[Ak:N,Rk,Rk+l] 

Ak:N^Ak:N 

Yk+i^y 

subject to Ak+i:N{t) G P/c+i, t G [0,T] a.e. 

Yk+i{t) G {d^k+i)i^k+i(t) + • • • + ANit)), t G [0, T] a.e., 


where 


Rk+i = -w 


fe+i 


/ {Ak+i + • ■ ■ An)Yi.+i — fik+i{Ak+i-.N)dt — / Yk+idXk+i, 
Jo Jo 


given Pfc+i C ^k+i : K —>■ M concave, and 


( 8 ) 


N 

Pk+i{Ak+i-.N) ■■= 

N 

Mvl+i := T'fe+ijWj 

j=k+l 


with 7 fc+i,fc+i := 1 and 7fc+i j := IlLfe+i Pe, j = k + 2,- ■ ■ , N. 

We next fix Rk-i and focus on the analogous optimization problem for Player k — 1. Player 
fc — 1 chooses {Rk, ■ ■ ■ , Rn) and {Ak-i, • • • , An) subject to incentive compatibility and individual 
rationality for Player k through Player N. By the inductive hypothesis, incentive compatibility and 
individual rationality for Players fc + 1, - • • , is equivalent to considering Rk+i of the form Q. 
Therefore, we can write the optimization problem of Player fc — 1 as 

sup j'^~^[Ak-i-.N,Rk-i,Rk] 

Ak-l:N^Ak-l:N 

Rk^Tlk 

subject to J^[Ak,N,Rk,Rk+i] > J^[A'k,N^Rk,Rk+i] 

'^■^'k-.N S Ak-.N U {Afc:7v : [0, T] —>■ I Ak-.N{t) G K X Pfe+i, t G [0, T] a.e.} 

Y[Ak-.N, Rk, Rk+l] > Wfc 

Ak:N{t) G M X Pfe-i-i, t G [0,T] a.e. 
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We note that it is now in the form of a generalized principal-agent problem. Furthermore, with this 
choice of Rk+i, we can re-write the utility for Player k as 


J^[Ak-.N, RkT Rk+l] {Ak)dt-\-Rk + [Rk+l] 

Jo 

= E^ [ r'^{Ak)dt-\-Rk-PkRk-i-i 

Jo 


= 


+ [ f^{Ak) A ldkP‘k+l{Ak+l-.N)dtRk 

Jo 


Define 


Pk{Ak:N) ■= r^iAk) PkPk+l{Ak+l-.N)- 
By Theorem [l] it is equivalent for Player fc — 1 to solve: 

sup J^~'^[Ak-l:N,Rk-l,Rk] 

Ak_i.pjGAk- 1:N 

Yk^y 

subject to Ak-.N(t) G P/c C t G [0,T] a.e. 

Yk{t) G {d^k){Ak{t) -f • • • -f A]k[{t)), t G [0, T] a.e., 


where ^ 

{AkAN)Yk — fJ,k{Ak:N)dt — / YkdXk, 

Jo 

4>k{Ak -I- • ■ ■ -I- An) is the {Ak -I- • • • -I- x P^+ij-concave envelope of pk, and Pfc C is 

the corresponding touching subset. 

By induction, we conclude that it is equivalent for Player 0 to solve the following optimization 
problem: 

sup J°[A, i?i] 

AeA,Yiey 

subject to A{t) G Pi C t G [0,r] a.e. 

Yi(t) G (9(^i)(Ai(f) -l- • • • -l- Ajv(t)), t G [0, T] a.e.. 


Rk ■■= wfc -h -t [ 
Jo 


where 


i?i =wt 


(^1 


+ An)Yi — pi{A)dt 



WdXi, 


where pi and w* are given by (§, (^1 is {Ai -|- • • • -I- Aat,® X r2)-concave envelope of pi, and 
Pi C is the corresponding touching set. □ 


Remark 1. Although we do not explicitly state the form of R 2 , ■ ■ ■ ,Rn in Theorem^ the details 
of the proof reveal that they will each be as follows: 

{Ak -I- • • • -I- Aat) Yk — pk {Ak:N) dt — f YkdXk 

Jo 



N 

Pk{Ak-N) ■='^lk,jr^{Aj), Wfe 
j^k 


N 

j = k 


where 
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with yk,k := 1; k = 1, - ■ ■ ,N and jij := Yli=i 1 < * < j < o.'nd 

Yk(t) S (d^k) {Ak{t) + • • • + AN(t )), (8) 

with : K —>■ IR defined in Theorem^ The particular choice ofY 2 ,--- , Yat will not ajfect the utility 
for Player 0, whenever Q holds, because it only depends upon Ri. The function pk represents the 
cost that Player k — 1 must pay for incentivizing the hierarchy from Player k to Player N to follow 
the suggested effort Ak-.N ■ The scalar constant represents a minimum expected utility that Player 
k—1 must provide to satisfy the individuality rationality constraints for Players k, ■ ■ ■ , N. By using 
the argument in Theorem ([^, we can confirm that Rk is (T)-measurable. Therefore, an optimal 
contract obtained by the proposed method is implementable. Furthermore, it is verifiable because an 
optimal Ak is adapted to . 

We refer to the components Ai,j^ and i? 2 :Af as ‘suggested’ efforts and compensations because 
Player 0 does not have to force the other players to choose them - we have shown it is optimal for 
players lower in the hierarchy to construct these contracts on their own. For this reason, we do not 
explicitly write them down in the theorem statement. 

At this point, Player 0 can solve this equivalent optimization problem via the dynamic pro¬ 
gramming principle and an associated Hamilton-Jacobi-Bellman equation. Player O’s optimization 
problem has a running cost associated with the choice of A and Y, as well as a terminal compen¬ 
sation associated with Ri. Then, it is natural to introduce a state variable W with the property 
that W{T) = Ri: 

Definition 5. Given a choice of processes { A,Y), we define the continuation value of the aggregate 
agents as 

[AiA]\[)Y — pi(^A)ds — f YdXi 

Jo 

= w* — / pi{A)ds — f aYdB. 

Jo Jo 


W{t) := wj + [ 
Jo 


Then, W{T) = i?i. 

Remark 2. The term ‘continuation value’ refers to the characterization ofWlf) as the expected 
remaining utility for Player 1 at time t conditioned on the information up to t, assuming he follows 
the suggested effort. In particular, we have 


W{t) = Ri 


pi{A)ds 


aYdB. 


Therefore, 


W{t) = 


J Pii^A^ds -\- i?i 


By viewing VF as a state variable of a (stochastic) dynamical system, we can now apply the 
dynamic programming principle to solve the reformulated optimization problem Q. 
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Theorem 3. Define the value function for Player 0 as 


v{w,t) := sup Ej 
AeA" 
Yey* 


oT ^ 


Y,Ak{s)dsPq\W{T))\W{t)=' 


fc=i 


Then, v is the unique viscosity solution to the following Hamilton-Jacobi-Bellman equation: 

T 


Vt + sup 
aG r 1 

ye{dfi)iC a) 


^y o Vww “t" 1 a 


= 0 , 


with terminal condition v{w,T) = q^(w). 

Proof. This is a standard dynamic programming argument which may be found, for example, in 

[SEl]. □ 

If there exists a smooth enough solution to this HJB, then by standard stochastic control 
arguments, we can construct optimal feedback controls: 


Ait) := A*iWit),t), Yit) := F*(IB(t),<). 

Then, using these feedback controls and the dynamics of Wit), ever player in the hierarchy can 
compute W in terms of observed values. Specifically, we have 

rt / N 


Wit) =<+I s)j F*(IB(s), s) - M (dl*(IB(s), s)) ds-J^ Y*iWis), s)dX, 

= <+/ Y*iWis),s)-fiiA*iWis),s))ds- j\*iWis),s)dX2 


AUWis),s)Y*iWis),s)-piiA*iWis),s))ds- [ Y*iWis),s)dXN 

Jo 

= wl- [ fiiA*iWis),s))ds- 
Jo 

Namely, all three processes - A, Y, and W - are adapted to and . Therefore, 

the proposed contract is implementable and verifiable. 

Player 0 then agrees to pay Player 1 an amount i?i = IT(T), where W is evolved via the 
optimal feedback controls. Player 0 announces the contract and suggested efforts and compensa¬ 
tions. Then, we have shown that it is optimal for the rest of the principal-agent hierarchy to follow 
A = (^ 1 , • • • , Ajv) and i? 2 :Ar = (.R 2 ,' • • j Pn)- 

One major advantage of this hierarchical model is the dimensionality of the resulting PDE 
- only one-dimensional in space. In comparison, the model with multiple agents reporting to 
a single principal in [5] resulted in a PDE with one state variable for each agent. Therefore, 
this hierarchical model is very tractable for even large X. It is worth noting, however, that this 
particular dimensionality result depend on the risk-neutrality of terminal payoffs between agents 


Y*iWis),s)dB. 
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in the hierarchy. Another important feature of the proposed approach is that Fi (and F/j’s) can 
be parameterized by a homeomorphism whose domain is one-dimensional when the players’ utility 
functions are strictly concave. This feature yields the maximization problem in the Hamiltonian 
of the HJB equation to be one-dimensional at each time. This additional dimensionality reduction 
result will be provided in Theorem]^ 

5 Details on (/, r)-Concave Envelopes 

In the previous iterative construction of an incentive compatible contract for the principal-agent 
hierarchy, the key condition for iteration is (/, F)-concavity. A simpler version of this condition was 
first highlighted in [5], but details were not provided on how to check (/, F)-concavity or construct 
appropriate envelopes. 

In this section, we provide a general construction of (/, F)-concave envelopes ^ of a function g, 
and also provide structural results about the subset where g = g. 

5.1 Constructing (/, r)-Concave Envelopes 

Recall from the definition in Section 3 that the key properties of the (/, F)-concave envelope are 
(1) /-concavity, (2) majorizing the function g over F, and (3) minimality. All three properties are 
intuitively represented by a step in the construction in this section. 

We construct the (/, F)-concave envelope of g in the following three steps: 

1. Define (j){y) := sup {g{x) | x G F s.t. f{x) = y}, where we set (/((y) := —oo if /“^({y})nF = 0. 

2. Compute the concave envelope (j) of (j), then 

3. Define g{x) = (j){f{x)). 

An example of the proposed construction is shown in Figure]^ 

Proposition 4. For any g : K™ —> K, the function g, as constructed above, is the {f,T)-concave 
envelope of g. 

Proof. By construction, g = (j)of is /-concave and majorizes g over F. Suppose there exists g = (fof 
which is /-concave, majorizes g over F, and is smaller than g at some point Xq € F, i.e., 

'^(/(a^o)) < <^(f(xo))- (9) 


Because g majorizes g over F, we have 

f>(f(x)) = sup {g(y) I y G F s.t. f(x) = f(y)} 

< sup{(f(f(y)) I y G F s.t. f(x) = f(y)} 

= Hfix)) Vx G F. 

Then, recalling that (j){y) = —oo when /“^({y}) C F = 0, we conclude that 

Hy) < Hy) Vy G M”, 

which contradicts ([^ because 0 is the concave envelope of 0. □ 
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Figure 2: An example of constructing (/, r)-concave envelope of g when f{x) = x. 


5.2 Sufficient Conditions for F to be One-Dimensional 

In this section we prove that if all players have utilities which are strictly concave in effort, then 
Fi C for Player 0 is parameterized by a homeomorphism i/; : M —>■ Fi. The signihcance of this 
result is that it provides dimensionality reduction in Player O’s optimization problem. In particular, 
rather than maximizing over a potentially A^-dimensional subset Fi C Player 0 can maximize 
over values in M using the map ip. In the lack of strict concavity, the dimension of Fi is not 
necessarily reduced. 

First, we prove an abstract result. 

Lemma 2. Let / : —)■ M 6e continuous. Suppose that x i— >■ f{x,y) is strictly concave for each y. 

Then, the function T : M —)■ M defined as 

T{y) := argmaxf{x,y) 

x^M. 


is continuous. 

Proof. By strict concavity of a; i—>■ f(x, y) for each y, we conclude that T is a well-defined function. 

Suppose that T is not continuous at y^. Then there exists e > 0 and a sequence ?/„ —>■ j/o such 
that |T(?/„) - T(?/o)| > e. Define 

5 ■= /(r( 2 /o),yo) - max{/(T(j/o) -f e, yo),/(T(j/o) - e,yo)} > 0, 

which is strictly positive by strict concavity in x. 
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By continuity of /, we can find ?/„ such that 

|T(y„)-T(yo)| >e 
\f{T{yo),yn) - /(r(j/o),yo)| < ^4 
IfiTiyo) + e,yn) - f{T{yo) + e,yo)\ < ^4 
\f{T{yo) - e, 2 /„) - f{T{yo) - e,yo)| < ^/4. 

Now, we verify two inequalities 

f{T{yo),yn)) - f{T{yo) + £,?/„) > f{T{yo),yo)) - /(T(yo) + e,yo) - S/2 

>5-5/2 = 5/2 > 0, 

f{T{yo),yn)) - f{T{yo) - > /(T(yo), 2/o)) - /('^(yo) - e,yo) - ^2 

>5-5/2 = 5/2 > 0. 

But by strict concavity of x i—>■ f{x,yn), this implies 

argmax/(x,T(y„)) e [T{yo) - e,T(yo) + e], 

which contradicts — T{yo)\ > e. □ 

Next, we prove the main argument which will be iterated. 

Lemma 3. Let g : M" -> K continuous. Let F C be parameterized by a homeomorphism 
■0 : K —>■ r such that 1^0(y) = y for all y G M.. Suppose that (y,z) i—>■ y(z,'0(y — z)) is jointly 
concave and, further, that z i—>■ g{z, ip{y — z)) is strictly concave for all y gM.. Let /(xi, • • • , x„) := 
xi + • • • + x„, g be the (/, K x r)-conca?;e envelope of g, and the corresponding touching set he 

r' := {(a:i, • • • , a:„) e M X r I g(xi, • • • , x„) = y(xi, • • • , x„)} . 

Then, F' is parameterized by a homeomorphism 0' : K —> F' such that 1^0'(y) = y for all y S K. 
Furthermore, y i—>■ giif'/y)) is concave. 

Proof. We note that for every y € M, we can parameterize the set 


{(xi, • • • , x„) e K X F I /(xi, • • • , x„) = y} c K” 


by the map x i—)• (z, ^(y 
that the map 


is continuous. 

Next, we compute 


z)). By the strict concavity of this map, we apply Lemmaj^to conclude 
T(y) := argmaxy {z,il}{y - z)) 

zGR 


0(y) := sup{y(xi, • • • ,x„) | (xi, • • • ,x„) G M x F, /(xi, • • • ,x„) = y} 
= sup {g (z, 0(y - z)) | z G K} . 


By strict concavity of z i—>• g[z,i/!{y — z)), we conclude that the supremum is attained at a single 
point for each y. The pointwise supremum of the jointly concave map (y, z) i—>■ y(z, 0(y — z)) is 
concave, as shown in [3], so y i—>■ 0(y) is concave. Therefore, 0 = 0, and y = 0 o /• 
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Define a new function -0' : K —>■ K x F as: 

By continuity of T, tp' is continuous. We can also check, (j){y) = g{ip'{y)), soy ^ gi'^'iv)) is concave 
for all y gM.. 

We claim that the image of ip' is F'. By definition, 

F' := {(a;i, • • • , x„) G K x F | g{xi, ■■■ , a:„) = g{xi, • • • , Xn)} 

= {{xi,--- , x„) G K X F I g{xi, • • • ,Xn) = 4>o f{xi,- ■ ■ , a;„)} 

= U • ,a;n) G M X F, f{xi, ■■■ ,Xn) = y\ g{xi,-- ■ ,a;„) = Piy)} 

ym 

= [jmy),iPiy-T{y)))} 

yGK 

= lm('0'). 

Finally, we confirm that 

l^iP'(y) = T{y) + l^ip{y - T{y)) = y, 

which implies ip' is one-to-one and an open map, so we conclude ■!/)' : K —>■ F' is a homeomorphism. 

□ 

The intuition of the following main result is to apply Lemma iteratively. At each step, 
g represents the effective utility of Player k over A^..pf. By induction, we assume F^-^i is one¬ 
dimensional and parametrized by a homeomorphism ipk+ij so Player k effectively maximizes over 
K^. We show under concavity assumptions that Ffc is then once again one-dimensional. 

Theorem 4. If each player’s utility function is strictly concave and (3k > 0, then Fi C is 
parameterized by a homeomorphism ipi : K —> Fi. 

Proof. First, recall from the iteration argument that at each stage, pk ■ —)■ K is a linear 

combination of r^, • • • ,r^ with positive coefficients. Furthermore, at each step, Ffc is the touching 
set corresponding to the {Ak -|- • • • -I- Ajv,K x Ffc_|_i)-concave envelope of pk- 

We want to show that for each fc, F^ is parameterized by a homeomorphism ipk : K —t F^ such 
that ipk{y) = y and y pk{ipk{y)) is concave. We proceed by induction on k. 

In the base case, we know Fjv = M, and this can be parameterized by the identity map ipN{y) = y- 
The concavity condition holds by concavity of r^. 

Now, consider the Player k’s effective utility function, which is given by 

Mfe(Afc:7v) = r’^{Ak) plk+iPk+i{Ak+i:N)- 

By the inductive hypothesis, there is a bijection ipk+i '■ K —t F/c+i such that 1^ipk+i{y) = y and 
y pk+iiipk+i{y)) is concave. 

In particular, we can confirm that 

Pkiz,ipk+iiy - z)) = r''{z) -G pk+i{ipk+i{y - z)) 

is strictly concave in z and jointly concave in (y,z). The right-hand-side is jointly concave in 
(y, z) as the composition of an affine function with pk+i o 'fJk+ii which is concave by the inductive 
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hypothesis. Because is strictly concave, we determine the sum is strictly concave in 0 and jointly 
concave in {y, z). 

Furthermore, by the inductive hypothesis tpk+iiy) = y- Therefore, we can apply Lemma 
to conclude the (Af^ + • • • + x Ffe+ij-concave envelope of pk has a corresponding touching 

set Ffc which is parameterized by a homeomorphism ijjk : K —)■ F^ such that = y and 

y t hki'ipkiy)) is concave. Then, the result follows by induction. □ 

The significance of this result is in reducing dimensionality in Player O’s optimization problem 
at each time step of dynamic programming from potentially iV-dimensional to one-dimensional. In 
particular. Player 0 can maximize any function (/> : F 1 —>■ M by maximizing cp o ipi : M —>■ M. 

Practically, this result means that in the case of strictly concave utilities, not only is the HJB 
PDF only one-dimensional in space, but the maximization step for computing an optimal Hamil¬ 
tonian at each time is also one-dimensional. This means that once ipi is computed. Player O’s 
optimal dynamic contract can be constructed in time that does not depend upon the size of the 
hierarchy. This result makes the model tractable even for very large N and contributes significantly 
to the tractability of using dynamic programming to design an optimal contract in a large-scale 
principal-agent hierarchy. 

6 Example: Quadratic Utilities 

In this section, we work through a specific example, where all agents have quadratic utilities. In 
particular, we set 

q^ir) := -hr 

Wk ■■= 0 

with /3k > 0 and > 0 for all 1 < fc < Finally, we choose 

■=-lay- 

Economically, this means each player 1 < fc < has an increasing aversion to putting in effort 
personally, which is governed by the parameter ak- Furthermore, each player 1 < fc < A^ has some 
aversion to paying the player below him in the hierarchy, which is governed by the parameter /3fc. 
Player 0 is averse to uncertainty in the terminal payment to players below him. 

In order to construct an optimal dynamic contract, we need to explicitly perform the iterative 
construction of Section 4. Because each utility function is assumed strictly concave, we can also 
apply the results of Section 5 to obtain a homeomorphism ipk : ffi —>■ F^ at each step. In the 
quadratic utility case, it will turn out that each F^ will be a one-dimensional linear subspace. We 
will see that the iteration process only requires linear algebraic computations at each step. 

Theorem 5. There exists a strictly positive-definite, diagonal matrix Ax, a vector gi with \ ^ gx = 1, 
and a real number wf such that it is equivalent for Player 0 to solve the following optimization 
problem: 

sup J^[A,Rx\ 

AcA,Yicy 

subject to A{t) € Fi C t G [0,r] a.e. 

Yx{t) G {dfix) (4fi(t) -b • • • -b t G [0, T] a.e., 
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with the compensation scheme given by 

R\ '.= Wi + f (^1 + • • ■ + Ajsf) Yi — p,i (A) dt 

Jo 



YidXi 


where: 

• pi{x) = —Jx^Aix, 

• 4>i{y) = -\9i , so d${y) = -gjAigiy, and 

• Fi is a one-dimensional subspace parametrized the isomorphism ipi^y) = giy. 


Proof. We follow the iteration argument of Theorems and to construct each pk, Ffc, and the 
homeomorphism : K —>■ Ffc. For Player N, we trivially have that p.n{x) = —Fjv = K, 
and the homeomorphism is ip m : K —>■ Fjv is the identity ipN{y) = y- 

Consider Player k with k > N. We know by Theorems]^ andthat there exist pk+i '■ —)■ 

Ffc+i C and a homeomorphism ipk+i ■ K —>■ Ffc+i such that ipk+i{y) = V- 

We suppose further that there exists a strictly positive-definite matrix and vector gu+i 

with the property gk+i = 1 such that /rfc_|_i(x) = — and ipk+i{y) = gk+iy- 

Then by Theorem pk '■ —>■ K can be written: 


yk{x,y) = r''(x) -k l3k+iPk-i-i{y) 


X 

T 

■ 0 ■ 


X 

. y. 


0 Pk+l^k+l 


. y. 


or in the form pCx) = —^x^AkX for a strictly positive-definite, diagonal matrix. 

By Theorem]^ we need to compute the functions Tk : R —>■ K, '0fe : R R^“^ and (/)fe : R —>■ R. 
We can compute: 


Tk{y) ■■= arginaxpk{z,ipk-\-i{y - z)) 

zGR 


= arg max 


- ^lJk+i{y - z)'^gJ_^_J^Ak+lgk-^-l 


/3fc-ngfc+iAfc+igfc+i 
Oil + l3k+igJ+iAk+igk+i' 


and then immediately, we have 

^k{y) ■■= {Tkiy),ipk+i{y - Tk{y))) = gky 

for a vector gk computable in terms of Tk and ipk+i- Further, we have y = ipk{y) 

9k = 1- 

Lastly, we have 

pk{y) ■■= suppkiz,ipk+i{y - z)) = Pkii’kiy)) = -^y'^gjJ^kgk- 

zGR z 


gky, so 


By induction, the result follows. We note that because '!/;i : R —>■ Fi is linear, then Fi is a 
one-dimensional linear subspace. □ 
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Then, as in Theorem Player 0 can define a value function and characterize it as the unique 
viscosity solution to an HJB equation. However, in this case, the HJB equation simplifies consid¬ 
erably: 

Corollary 1. Define the value function v for Player 0 as in Theorem^ Then v is the unique 
viscosity solution to the Hamilton-Jacobi-Bellman equation: 


Vt -\- sup 

yCR 


'1 

.2 


(ycT Vww 


+ Vw)y^ Py 


= 0 , 


with terminal condition v{w,T) = q^{w) and 7 := gjAigi. 

The proof is just a particular case of Theorem]^ The significance is that, in this case, the HJB 
reduces to only one state variable and a one-dimensional maximization over M at each time. 
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